Papers with partially noising model structure
InfoDiffusion: Information Entropy Aware Diffusion Process for Non-Autoregressive Text Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing text diffusion models have failed to capture the difference between the “easy-first” process and the “keyword-first" process of humans. |
| Approach: | They propose a non-autoregressive text diffusion model that incorporates a "keyinfo-first" generation strategy and a noise schedule based on the amount of text information. |
| Outcome: | The proposed model outperforms the baseline model in terms of generation quality and diversity, and higher sampling efficiency. |